Hardness on Numerical Realization of Some Penalized Likelihood Estimators
نویسندگان
چکیده
Abstract: We show that with a class of penalty functions, numerical problems associated with the implementation of the penalized least square estimators are equivalent to the exact cover by 3-sets problem, which belongs to a class of NP-hard problems. We then extend this NP-hardness result to the cases of penalized least absolute deviation regression and penalized support vector machines. We discuss the practical implication of our results. In particular, we emphasize that the oracle property of a penalized likelihood estimator requires a local extremum, instead of a global extremum. Hence the penalized likelihood estimators are still favorable; however one should not attempt to find its global extremum(a)!
منابع مشابه
Penalized Estimators in Cox Regression Model
The proportional hazard Cox regression models play a key role in analyzing censored survival data. We use penalized methods in high dimensional scenarios to achieve more efficient models. This article reviews the penalized Cox regression for some frequently used penalty functions. Analysis of medical data namely ”mgus2” confirms the penalized Cox regression performs better than the cox regressi...
متن کاملMean and variance of implicitly defined biased estimators (such as penalized maximum likelihood): applications to tomography
Many estimators in signal processing problems are defined implicitly as the maximum of some objective function. Examples of implicitly defined estimators include maximum likelihood, penalized likelihood, maximum a posteriori, and nonlinear least squares estimation. For such estimators, exact analytical expressions for the mean and variance are usually unavailable. Therefore, investigators usual...
متن کاملMean and Variance of Implicitly Defined Biased Estimators (Such as Penalized Maximum Likelihood): Ap - Image Processing, IEEE Transactions on
Many estimators in signal processing problems are defined implicitly as the maximum of some objective function. Examples of implicitly defined estimators include maximum likelihood, penalized likelihood, maximum a posteriori, and nonlinear least squares estimation. For such estimators, exact analytical expressions for the mean and variance are usually unavailable. Therefore, investigators usual...
متن کاملModified Maximum Likelihood Estimation in First-Order Autoregressive Moving Average Models with some Non-Normal Residuals
When modeling time series data using autoregressive-moving average processes, it is a common practice to presume that the residuals are normally distributed. However, sometimes we encounter non-normal residuals and asymmetry of data marginal distribution. Despite widespread use of pure autoregressive processes for modeling non-normal time series, the autoregressive-moving average models have le...
متن کاملChange-point estimators with true identification property
The change-point problem is reformulated as a penalized likelihood estimation problem. A new non-convex penalty function is introduced to allow consistent estimation of the number of change points, and their locations and sizes. Penalized likelihood methods based on LASSO and SCAD penalties may not satisfy such a property. The asymptotic properties for the local solutions are established and nu...
متن کامل